Method and system for improving 2d-3d registration convergence

ABSTRACT

A method for registration of digital medical images is provided. The method includes the step of storing a 3D digital medical image having a 3D anatomical feature and a first coordinate system and storing a 2D digital medical image having a 2D anatomical feature and a second coordinate system. The method further includes the steps of storing a placement of a digital medical object on the 3D digital medical image and the 2D digital medical image and generating a simulated 2D digital medical image from the 3D digital medical image, wherein the simulated 2D digital medical image comprises a simulated 2D anatomical feature corresponding to the 3D anatomical feature. The 2D anatomical feature is compared with the simulated 2D anatomical feature until a match is reached and a registration of the first coordinate system with the second coordinate system based on the match is determined.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/289,537 which is a continuation-in-part of U.S. patent applicationSer. No. 15/157,444 filed May 18, 2016, which is a continuation-in-partof U.S. patent application Ser. No. 15/095,883, filed Apr. 11, 2016,which is a continuation-in-part of U.S. patent application Ser. No.14/062,707, filed on Oct. 24, 2013, which is a continuation-in-partapplication of U.S. patent application Ser. No. 13/924,505, filed onJun. 21, 2013, which claims priority to provisional application No.61/662,702 filed on Jun. 21, 2012 and claims priority to provisionalapplication No. 61/800,527 filed on Mar. 15, 2013, all of which areincorporated by reference herein in their entireties for all purposes.

FIELD

The present disclosure relates to position recognition systems, and inparticular, multi-image registration for robot assisted surgery.

BACKGROUND

In the field of image guidance, registration is the quantification ofthe transformation between two or more coordinate systems. Aftersuccessful registration, the position of a tool or other object in onecoordinate system, such as an optically tracked space, can be accuratelydisplayed in another coordinate system, such as the medical image space.In the case where image guidance or robot-assisted image guidance is tobe performed using a preoperative 3D image dataset such as a computedtomography (CT) scan or magnetic resonance imaging (MRI) scan,co-registration among multiple coordinate systems may be needed, such asbetween a preoperatively obtained anatomical CT or MRI coordinatesystem, an intraoperatively obtained anatomical coordinate system, acoordinate system of the tracking cameras, and the like.

One way to achieve co-registration of multiple coordinate systems is touse 2D-3D registration, such as where a pair of 2D x-ray radiographs ofthe patient is taken at the time of surgery, with the position of thex-ray machine and patient tracked using tracking cameras. The coordinatesystem in which the x-rays are taken may then be registered to apreoperatively obtained 3D medical image coordinate system throughmethods of 2D-3D registration. In this method, the 3D CT or MM datasetmay be used to generate 2D reconstructed planar images simulating x-rayradiographs. One way to generate 2D reconstructed simulated x-ray imagesfrom a 3D dataset is to trace and integrate the intensities along raysfrom a point source projected through the volumetric medical image on a2D plane (e.g., a digitally reconstructed radiograph (DRR)). The DRRsare generated iteratively until they match the actual 2D x-ray images;that is, until the features or intensity characteristics of the bonestructures on the DRRs and actual radiographs overlap within sometolerance. For instance, the iterative method could be a method such asPowell's Method, by which a cost function is minimized by starting witha guess and then adjusting parameters systematically until the error iswithin tolerance. As an example, the cost function could be constructedby subtracting the pixel intensities at locations within the images inthe DRRs and the actual x-ray radiographs, and would be minimized whenthe pixel intensities agreed closest between X-ray and DRR in both viewsof the x-ray pair. Parameters of the cost function that could beadjusted between iterations may include the position and orientation ofthe 3D volumetric data, the x-ray source, angles of x-ray paths relativeto the 3D volume, and the like, varied independently and/orsimultaneously within the known (tracked) geometric constraint of theactual relative positions of the x-ray machine when the pair of shotswere taken. Once a match is found, the position in the CT or MMcoordinate system in which the x-ray machine must have been at the timethe x-rays were taken is known from the parameters used in thecalculation. Also, the position of the actual x-ray machine in thetracking coordinate system is known from tracking cameras. Therefore,the transformations between CT (or MRI), x-ray, and camera coordinatesystems are determined.

Iterative methods as mentioned above, however, may be problematicbecause a large number of iterations may be required before a successfulmatch is found. This may result in a long time delay, or worse, theiterations may fail to converge on a solution. Therefore, systems andmethods are needed to improve the convergence of 2D-3D registration.

SUMMARY

The present disclosure provides methods and systems that improve 2D-3Dregistration convergence by initializing the computational configurationsuch that the simulated and actual x-rays agree fairly well beforestarting iterations. Improvements may result in less iteration, decreaseprocessing time, lower incidence of failure to converge, and the like.

In one embodiment, there is provided a system and method forregistration of digital medical images. The method includes the step ofstoring a 3D digital medical image having a 3D anatomical feature and afirst coordinate system and storing a 2D digital medical image having a2D anatomical feature and a second coordinate system. The method furtherincludes the steps of storing a placement of a digital medical object onthe 3D digital medical image and the 2D digital medical image andgenerating a simulated 2D digital medical image from the 3D digitalmedical image, wherein the simulated 2D digital medical image comprisesa simulated 2D anatomical feature corresponding to the 3D anatomicalfeature. The 2D anatomical feature is compared with the simulated 2Danatomical feature until a match is reached and a registration of thefirst coordinate system with the second coordinate system based on thematch is determined.

These and other systems, methods, objects, features, and advantages ofthe present invention will be apparent to those skilled in the art fromthe following detailed description of the preferred embodiment and thedrawings. All documents mentioned herein are hereby incorporated intheir entirety by reference.

BRIEF DESCRIPTION OF THE FIGURES

The invention and the following detailed description of certainembodiments thereof may be understood by reference to the followingfigures:

FIG. 1 is an overhead view of a potential arrangement for locations ofthe robotic system, patient, surgeon, and other medical personnel duringa surgical procedure;

FIG. 2 illustrates the robotic system including positioning of thesurgical robot and the camera relative to the patient according to oneembodiment;

FIG. 3 illustrates a surgical robotic system in accordance with anexemplary embodiment;

FIG. 4 illustrates a portion of a surgical robot in accordance with anexemplary embodiment;

FIG. 5 illustrates a block diagram of a surgical robot in accordancewith an exemplary embodiment;

FIG. 6 illustrates a surgical robot in accordance with an exemplaryembodiment;

FIGS. 7A-7C illustrate an end effector in accordance with an exemplaryembodiment;

FIG. 8 illustrates a surgical instrument and the end effector, beforeand after, inserting the surgical instrument into the guide tube of theend effector according to one embodiment;

FIGS. 9A-9C illustrate portions of an end effector and robot arm inaccordance with an exemplary embodiment;

FIG. 10 illustrates a dynamic reference array, an imaging array, andother components in accordance with an exemplary embodiment;

FIG. 11 illustrates a method of registration in accordance with anexemplary embodiment;

FIG. 12A-12B illustrate embodiments of imaging devices according toexemplary embodiments;

FIG. 13 illustrates a process of planning the trajectory of a pediclescrew on a single lumbar vertebra in multiple slices through a computedtomography volume.

FIG. 14 illustrates a process of planning the trajectory of a pediclescrew on a single lumbar vertebra from multiple views on an x-ray.

FIG. 15 illustrates an exemplary method consistent with the presentdisclosure.

While the invention has been described in connection with certainpreferred embodiments, other embodiments would be understood by one ofordinary skill in the art and are encompassed herein.

DETAILED DESCRIPTION

It is to be understood that the present disclosure is not limited in itsapplication to the details of construction and the arrangement ofcomponents set forth in the description herein or illustrated in thedrawings. The teachings of the present disclosure may be used andpracticed in other embodiments and practiced or carried out in variousways. Also, it is to be understood that the phraseology and terminologyused herein is for the purpose of description and should not be regardedas limiting. The use of “including,” “comprising,” or “having” andvariations thereof herein is meant to encompass the items listedthereafter and equivalents thereof as well as additional items. Unlessspecified or limited otherwise, the terms “mounted,” “connected,”“supported,” and “coupled” and variations thereof are used broadly andencompass both direct and indirect mountings, connections, supports, andcouplings. Further, “connected” and “coupled” are not restricted tophysical or mechanical connections or couplings.

The following discussion is presented to enable a person skilled in theart to make and use embodiments of the present disclosure. Variousmodifications to the illustrated embodiments will be readily apparent tothose skilled in the art, and the principles herein can be applied toother embodiments and applications without departing from embodiments ofthe present disclosure. Thus, the embodiments are not intended to belimited to embodiments shown, but are to be accorded the widest scopeconsistent with the principles and features disclosed herein. Thefollowing detailed description is to be read with reference to thefigures, in which like elements in different figures have like referencenumerals. The figures, which are not necessarily to scale, depictselected embodiments and are not intended to limit the scope of theembodiments. Skilled artisans will recognize the examples providedherein have many useful alternatives and fall within the scope of theembodiments.

Turning now to the drawing, FIGS. 1 and 2 illustrate a surgical robotsystem 100 in accordance with an exemplary embodiment. Surgical robotsystem 100 may include, for example, a surgical robot 102, one or morerobot arms 104, a base 106, a display 110, an end effector 112, forexample, including a guide tube 114, and one or more tracking markers118. The surgical robot system 100 may include a patient tracking device116 also including one or more tracking markers 118, which is adapted tobe secured directly to the patient 210 (e.g., to the bone of the patient210). The surgical robot system 100 may also utilize a camera 200, forexample, positioned on a camera stand 202. The camera stand 202 can haveany suitable configuration to move, orient, and support the camera 200in a desired position. The camera 200 may include any suitable camera orcameras, such as one or more infrared cameras (e.g., bifocal orstereophotogrammetric cameras), able to identify, for example, activeand passive tracking markers 118 in a given measurement volume viewablefrom the perspective of the camera 200. The camera 200 may scan thegiven measurement volume and detect the light that comes from themarkers 118 in order to identify and determine the position of themarkers 118 in three dimensions. For example, active markers 118 mayinclude infrared-emitting markers that are activated by an electricalsignal (e.g., infrared light emitting diodes (LEDs)), and passivemarkers 118 may include retro-reflective markers that reflect infraredlight (e.g., they reflect incoming IR radiation into the direction ofthe incoming light), for example, emitted by illuminators on the camera200 or other suitable device.

FIGS. 1 and 2 illustrate a potential configuration for the placement ofthe surgical robot system 100 in an operating room environment. Forexample, the robot 102 may be positioned near or next to patient 210.Although depicted near the head of the patient 210, it will beappreciated that the robot 102 can be positioned at any suitablelocation near the patient 210 depending on the area of the patient 210undergoing the operation. The camera 200 may be separated from the robotsystem 100 and positioned at the foot of patient 210. This locationallows the camera 200 to have a direct visual line of sight to thesurgical field 208. Again, it is contemplated that the camera 200 may belocated at any suitable position having line of sight to the surgicalfield 208. In the configuration shown, the surgeon 120 may be positionedacross from the robot 102, but is still able to manipulate the endeffector 112 and the display 110. A surgical assistant 126 may bepositioned across from the surgeon 120 again with access to both the endeffector 112 and the display 110. If desired, the locations of thesurgeon 120 and the assistant 126 may be reversed. The traditional areasfor the anesthesiologist 122 and the nurse or scrub tech 124 remainunimpeded by the locations of the robot 102 and camera 200.

With respect to the other components of the robot 102, the display 110can be attached to the surgical robot 102 and in other exemplaryembodiments, display 110 can be detached from surgical robot 102, eitherwithin a surgical room with the surgical robot 102, or in a remotelocation. End effector 112 may be coupled to the robot arm 104 andcontrolled by at least one motor. In exemplary embodiments, end effector112 can comprise a guide tube 114, which is able to receive and orient asurgical instrument 608 (described further herein) used to performsurgery on the patient 210. As used herein, the term “end effector” isused interchangeably with the terms “end-effectuator” and “effectuatorelement.” Although generally shown with a guide tube 114, it will beappreciated that the end effector 112 may be replaced with any suitableinstrumentation suitable for use in surgery. In some embodiments, endeffector 112 can comprise any known structure for effecting the movementof the surgical instrument 608 in a desired manner.

The surgical robot 102 is able to control the translation andorientation of the end effector 112. The robot 102 is able to move endeffector 112 along x-, y-, and z-axes, for example. The end effector 112can be configured for selective rotation about one or more of the x-,y-, and z-axis, and a Z Frame axis (such that one or more of the EulerAngles (e.g., roll, pitch, and/or yaw) associated with end effector 112can be selectively controlled). In some exemplary embodiments, selectivecontrol of the translation and orientation of end effector 112 canpermit performance of medical procedures with significantly improvedaccuracy compared to conventional robots that utilize, for example, asix degree of freedom robot arm comprising only rotational axes. Forexample, the surgical robot system 100 may be used to operate on patient210, and robot arm 104 can be positioned above the body of patient 210,with end effector 112 selectively angled relative to the z-axis towardthe body of patient 210.

In some exemplary embodiments, the position of the surgical instrument608 can be dynamically updated so that surgical robot 102 can be awareof the location of the surgical instrument 608 at all times during theprocedure. Consequently, in some exemplary embodiments, surgical robot102 can move the surgical instrument 608 to the desired position quicklywithout any further assistance from a physician (unless the physician sodesires). In some further embodiments, surgical robot 102 can beconfigured to correct the path of the surgical instrument 608 if thesurgical instrument 608 strays from the selected, preplanned trajectory.In some exemplary embodiments, surgical robot 102 can be configured topermit stoppage, modification, and/or manual control of the movement ofend effector 112 and/or the surgical instrument 608. Thus, in use, inexemplary embodiments, a physician or other user can operate the system100, and has the option to stop, modify, or manually control theautonomous movement of end effector 112 and/or the surgical instrument608. Further details of surgical robot system 100 including the controland movement of a surgical instrument 608 by surgical robot 102 can befound in co-pending U.S. patent application Ser. No. 13/924,505, whichis incorporated herein by reference in its entirety.

The robotic surgical system 100 can comprise one or more trackingmarkers 118 configured to track the movement of robot arm 104, endeffector 112, patient 210, and/or the surgical instrument 608 in threedimensions. In exemplary embodiments, a plurality of tracking markers118 can be mounted (or otherwise secured) thereon to an outer surface ofthe robot 102, such as, for example and without limitation, on base 106of robot 102, on robot arm 104, or on the end effector 112. In exemplaryembodiments, at least one tracking marker 118 of the plurality oftracking markers 118 can be mounted or otherwise secured to the endeffector 112. One or more tracking markers 118 can further be mounted(or otherwise secured) to the patient 210. In exemplary embodiments, theplurality of tracking markers 118 can be positioned on the patient 210spaced apart from the surgical field 208 to reduce the likelihood ofbeing obscured by the surgeon, surgical tools, or other parts of therobot 102. Further, one or more tracking markers 118 can be furthermounted (or otherwise secured) to the surgical tools 608 (e.g., a screwdriver, dilator, implant inserter, or the like). Thus, the trackingmarkers 118 enable each of the marked objects (e.g., the end effector112, the patient 210, and the surgical tools 608) to be tracked by therobot 102. In exemplary embodiments, system 100 can use trackinginformation collected from each of the marked objects to calculate theorientation and location, for example, of the end effector 112, thesurgical instrument 608 (e.g., positioned in the tube 114 of the endeffector 112), and the relative position of the patient 210.

In exemplary embodiments, one or more of markers 118 may be opticalmarkers. In some embodiments, the positioning of one or more trackingmarkers 118 on end effector 112 can maximize the accuracy of thepositional measurements by serving to check or verify the position ofend effector 112. Further details of surgical robot system 100 includingthe control, movement and tracking of surgical robot 102 and of asurgical instrument 608 can be found in co-pending U.S. patentapplication Ser. No. 13/924,505, which is incorporated herein byreference in its entirety.

Exemplary embodiments include one or more markers 118 coupled to thesurgical instrument 608. In exemplary embodiments, these markers 118,for example, coupled to the patient 210 and surgical instruments 608, aswell as markers 118 coupled to the end effector 112 of the robot 102 cancomprise conventional infrared light-emitting diodes (LEDs) or anOptotrak® diode capable of being tracked using a commercially availableinfrared optical tracking system such as Optotrak®. Optotrak® is aregistered trademark of Northern Digital Inc., Waterloo, Ontario,Canada. In other embodiments, markers 118 can comprise conventionalreflective spheres capable of being tracked using a commerciallyavailable optical tracking system such as Polaris Spectra. PolarisSpectra is also a registered trademark of Northern Digital, Inc. In anexemplary embodiment, the markers 118 coupled to the end effector 112are active markers which comprise infrared light-emitting diodes whichmay be turned on and off, and the markers 118 coupled to the patient 210and the surgical instruments 608 comprise passive reflective spheres.

In exemplary embodiments, light emitted from and/or reflected by markers118 can be detected by camera 200 and can be used to monitor thelocation and movement of the marked objects. In alternative embodiments,markers 118 can comprise a radio-frequency and/or electromagneticreflector or transceiver and the camera 200 can include or be replacedby a radio-frequency and/or electromagnetic transceiver.

Similar to surgical robot system 100, FIG. 3 illustrates a surgicalrobot system 300 and camera stand 302, in a docked configuration,consistent with an exemplary embodiment of the present disclosure.Surgical robot system 300 may comprise a robot 301 including a display304, upper arm 306, lower arm 308, end effector 310, vertical column312, casters 314, cabinet 316, tablet drawer 318, connector panel 320,control panel 322, and ring of information 324. Camera stand 302 maycomprise camera 326. These components are described in greater withrespect to FIG. 5. FIG. 3 illustrates the surgical robot system 300 in adocked configuration where the camera stand 302 is nested with the robot301, for example, when not in use. It will be appreciated by thoseskilled in the art that the camera 326 and robot 301 may be separatedfrom one another and positioned at any appropriate location during thesurgical procedure, for example, as shown in FIGS. 1 and 2. FIG. 4illustrates a base 400 consistent with an exemplary embodiment of thepresent disclosure. Base 400 may be a portion of surgical robot system300 and comprise cabinet 316. Cabinet 316 may house certain componentsof surgical robot system 300 including but not limited to a battery 402,a power distribution module 404, a platform interface board module 406,a computer 408, a handle 412, and a tablet drawer 414. The connectionsand relationship between these components is described in greater detailwith respect to FIG. 5.

FIG. 5 illustrates a block diagram of certain components of an exemplaryembodiment of surgical robot system 300. Surgical robot system 300 maycomprise platform subsystem 502, computer subsystem 504, motion controlsubsystem 506, and tracking subsystem 532. Platform subsystem 502 mayfurther comprise battery 402, power distribution module 404, platforminterface board module 406, and tablet charging station 534. Computersubsystem 504 may further comprise computer 408, display 304, andspeaker 536. Motion control subsystem 506 may further comprise drivercircuit 508, motors 510, 512, 514, 516, 518, stabilizers 520, 522, 524,526, end effector 310, and controller 538. Tracking subsystem 532 mayfurther comprise position sensor 540 and camera converter 542. System300 may also comprise a foot pedal 544 and tablet 546.

Input power is supplied to system 300 via a power source 548 which maybe provided to power distribution module 404. Power distribution module404 receives input power and is configured to generate different powersupply voltages that are provided to other modules, components, andsubsystems of system 300. Power distribution module 404 may beconfigured to provide different voltage supplies to platform interfacemodule 406, which may be provided to other components such as computer408, display 304, speaker 536, driver 508 to, for example, power motors512, 514, 516, 518 and end effector 310, motor 510, ring 324, cameraconverter 542, and other components for system 300 for example, fans forcooling the electrical components within cabinet 316.

Power distribution module 404 may also provide power to other componentssuch as tablet charging station 534 that may be located within tabletdrawer 318. Tablet charging station 534 may be in wireless or wiredcommunication with tablet 546 for charging table 546. Tablet 546 may beused by a surgeon consistent with the present disclosure and describedherein. Power distribution module 404 may also be connected to battery402, which serves as temporary power source in the event that powerdistribution module 404 does not receive power from input power 548. Atother times, power distribution module 404 may serve to charge battery402 if necessary.

Other components of platform subsystem 502 may also include connectorpanel 320, control panel 322, and ring 324. Connector panel 320 mayserve to connect different devices and components to system 300 and/orassociated components and modules. Connector panel 320 may contain oneor more ports that receive lines or connections from differentcomponents. For example, connector panel 320 may have a ground terminalport that may ground system 300 to other equipment, a port to connectfoot pedal 544 to system 300, a port to connect to tracking subsystem532, which may comprise position sensor 540, camera converter 542, andcameras 326 associated with camera stand 302. Connector panel 320 mayalso include other ports to allow USB, Ethernet, HDMI communications toother components, such as computer 408.

Control panel 322 may provide various buttons or indicators that controloperation of system 300 and/or provide information regarding system 300.For example, control panel 322 may include buttons to power on or offsystem 300, lift or lower vertical column 312, and lift or lowerstabilizers 520-526 that may be designed to engage casters 314 to locksystem 300 from physically moving. Other buttons may stop system 300 inthe event of an emergency, which may remove all motor power and applymechanical brakes to stop all motion from occurring. Control panel 322may also have indicators notifying the user of certain system conditionssuch as a line power indicator or status of charge for battery 402.

Ring 324 may be a visual indicator to notify the user of system 300 ofdifferent modes that system 300 is operating under and certain warningsto the user.

Computer subsystem 504 includes computer 408, display 304, and speaker536. Computer 504 includes an operating system and software to operatesystem 300. Computer 504 may receive and process information from othercomponents (for example, tracking subsystem 532, platform subsystem 502,and/or motion control subsystem 506) in order to display information tothe user. Further, computer subsystem 504 may also include speaker 536to provide audio to the user.

Tracking subsystem 532 may include position sensor 504 and converter542. Tracking subsystem 532 may correspond to camera stand 302 includingcamera 326 as described with respect to FIG. 3. Position sensor 504 maybe camera 326. Tracking subsystem may track the location of certainmarkers that are located on the different components of system 300and/or instruments used by a user during a surgical procedure. Thistracking may be conducted in a manner consistent with the presentdisclosure including the use of infrared technology that tracks thelocation of active or passive elements, such as LEDs or reflectivemarkers, respectively. The location, orientation, and position ofstructures having these types of markers may be provided to computer 408which may be shown to a user on display 304. For example, a surgicalinstrument 608 having these types of markers and tracked in this manner(which may be referred to as a navigational space) may be shown to auser in relation to a three dimensional image of a patient's anatomicalstructure. Motion control subsystem 506 may be configured to physicallymove vertical column 312, upper arm 306, lower arm 308, or rotate endeffector 310. The physical movement may be conducted through the use ofone or more motors 510-518. For example, motor 510 may be configured tovertically lift or lower vertical column 312. Motor 512 may beconfigured to laterally move upper arm 308 around a point of engagementwith vertical column 312 as shown in FIG. 3. Motor 514 may be configuredto laterally move lower arm 308 around a point of engagement with upperarm 308 as shown in FIG. 3. Motors 516 and 518 may be configured to moveend effector 310 in a manner such that one may control the roll and onemay control the tilt, thereby providing multiple angles that endeffector 310 may be moved. These movements may be achieved by controller538 which may control these movements through load cells disposed on endeffector 310 and activated by a user engaging these load cells to movesystem 300 in a desired manner.

Moreover, system 300 may provide for automatic movement of verticalcolumn 312, upper arm 306, and lower arm 308 through a user indicatingon display 304 (which may be a touchscreen input device) the location ofa surgical instrument or component on three dimensional image of thepatient's anatomy on display 304. The user may initiate this automaticmovement by stepping on foot pedal 544 or some other input means.

FIG. 6 illustrates a surgical robot system 600 consistent with anexemplary embodiment. Surgical robot system 600 may comprise endeffector 602, robot arm 604, guide tube 606, instrument 608, and robotbase 610. Instrument tool 608 may be attached to a tracking array 612including one or more tracking markers (such as markers 118) and have anassociated trajectory 614. Trajectory 614 may represent a path ofmovement that instrument tool 608 is configured to travel once it ispositioned through or secured in guide tube 606, for example, a path ofinsertion of instrument tool 608 into a patient. In an exemplaryoperation, robot base 610 may be configured to be in electroniccommunication with robot arm 604 and end effector 602 so that surgicalrobot system 600 may assist a user (for example, a surgeon) in operatingon the patient 210. Surgical robot system 600 may be consistent withpreviously described surgical robot system 100 and 300.

A tracking array 612 may be mounted on instrument 608 to monitor thelocation and orientation of instrument tool 608. The tracking array 612may be attached to an instrument 608 and may comprise tracking markers804. As best seen in FIG. 8, tracking markers 804 may be, for example,light emitting diodes and/or other types of reflective markers (e.g.,markers 118 as described elsewhere herein). The tracking devices may beone or more line of sight devices associated with the surgical robotsystem. As an example, the tracking devices may be one or more cameras200, 326 associated with the surgical robot system 100, 300 and may alsotrack tracking array 612 for a defined domain or relative orientationsof the instrument 608 in relation to the robot arm 604, the robot base610, end effector 602, and/or the patient 210. The tracking devices maybe consistent with those structures described in connection with camerastand 302 and tracking subsystem 532.

FIGS. 7A, 7B, and 7C illustrate a top view, front view, and side view,respectively, of end effector 602 consistent with an exemplaryembodiment. End effector 602 may comprise one or more tracking markers702. Tracking markers 702 may be light emitting diodes or other types ofactive and passive markers, such as tracking markers 118 that have beenpreviously described. In an exemplary embodiment, the tracking markers702 are active infrared-emitting markers that are activated by anelectrical signal (e.g., infrared light emitting diodes (LEDs)). Thus,tracking markers 702 may be activated such that the infrared markers 702are visible to the camera 200, 326 or may be deactivated such that theinfrared markers 702 are not visible to the camera 200, 326. Thus, whenthe markers 702 are active, the end effector 602 may be controlled bythe system 100, 300, 600, and when the markers 702 are deactivated, theend effector 602 may be locked in position and unable to be moved by thesystem 100, 300, 600. Markers 702 may be disposed on or within endeffector 602 in a manner such that the markers 702 are visible by one ormore cameras 200, 326 or other tracking devices associated with thesurgical robot system 100, 300, 600. The camera 200, 326 or othertracking devices may track end effector 602 as it moves to differentpositions and viewing angles by following the movement of trackingmarkers 702. The location of markers 702 and/or end effector 602 may beshown on a display 110, 304 associated with the surgical robot system100, 300, 600, for example, display 110 as shown in FIG. 2 and/ordisplay 304 shown in FIG. 3. This display 110, 304 may allow a user toensure that end effector 602 is in a desirable position in relation torobot arm 604, robot base 610, the patient 210, and/or the user.

For example, as shown in FIG. 7A, markers 702 may be placed around thesurface of end effector 602 so that a tracking device placed away fromthe surgical field 208 and facing toward the robot 102, 301 and thecamera 200, 326 is able to view at least 3 of the markers 702 through arange of common orientations of the end effector 602 relative to thetracking device 100, 300, 600. For example, distribution of markers 702in this way allows end effector 602 to be monitored by the trackingdevices when end effector 602 is translated and rotated in the surgicalfield 208.

In addition, in exemplary embodiments, end effector 602 may be equippedwith infrared (IR) receivers that can detect when an external camera200, 326 is getting ready to read markers 702. Upon this detection, endeffector 602 may then illuminate markers 702. The detection by the IRreceivers that the external camera 200, 326 is ready to read markers 702may signal the need to synchronize a duty cycle of markers 702, whichmay be light emitting diodes, to an external camera 200, 326. This mayalso allow for lower power consumption by the robotic system as a whole,whereby markers 702 would only be illuminated at the appropriate timeinstead of being illuminated continuously. Further, in exemplaryembodiments, markers 702 may be powered off to prevent interference withother navigation tools, such as different types of surgical instruments608.

FIG. 8 depicts one type of surgical instrument 608 including a trackingarray 612 and tracking markers 804. Tracking markers 804 may be of anytype described herein including but not limited to light emitting diodesor reflective spheres. Markers 804 are monitored by tracking devicesassociated with the surgical robot system 100, 300, 600 and may be oneor more of the line of sight cameras 200, 326. The cameras 200, 326 maytrack the location of instrument 608 based on the position andorientation of tracking array 612 and markers 804. A user, such as asurgeon 120, may orient instrument 608 in a manner so that trackingarray 612 and markers 804 are sufficiently recognized by the trackingdevice or camera 200, 326 to display instrument 608 and markers 804 on,for example, display 110 of the exemplary surgical robot system.

The manner in which a surgeon 120 may place instrument 608 into guidetube 606 of the end effector 602 and adjust the instrument 608 isevident in FIG. 8. The hollow tube or guide tube 114, 606 of the endeffector 112, 310, 602 is sized and configured to receive at least aportion of the surgical instrument 608. The guide tube 114, 606 isconfigured to be oriented by the robot arm 104 such that insertion andtrajectory for the surgical instrument 608 is able to reach a desiredanatomical target within or upon the body of the patient 210. Thesurgical instrument 608 may include at least a portion of a generallycylindrical instrument. Although a screw driver is exemplified as thesurgical tool 608, it will be appreciated that any suitable surgicaltool 608 may be positioned by the end effector 602. By way of example,the surgical instrument 608 may include one or more of a guide wire,cannula, a retractor, a drill, a reamer, a screw driver, an insertiontool, a removal tool, or the like. Although the hollow tube 114, 606 isgenerally shown as having a cylindrical configuration, it will beappreciated by those of skill in the art that the guide tube 114, 606may have any suitable shape, size and configuration desired toaccommodate the surgical instrument 608 and access the surgical site.

FIGS. 9A-9C illustrate end effector 602 and a portion of robot arm 604consistent with an exemplary embodiment. End effector 602 may furthercomprise body 1202 and clamp 1204. Clamp 1204 may comprise handle 1206,balls 1208, spring 1210, and lip 1212. Robot arm 604 may furthercomprise depressions 1214, mounting plate 1216, lip 1218, and magnets1220. [0072] End effector 602 may mechanically interface and/or engagewith the surgical robot system and robot arm 604 through one or morecouplings. For example, end effector 602 may engage with robot arm 604through a locating coupling and/or a reinforcing coupling. Through thesecouplings, end effector 602 may fasten with robot arm 604 outside aflexible and sterile barrier. In an exemplary embodiment, the locatingcoupling may be a magnetically kinematic mount and the reinforcingcoupling may be a five bar over center clamping linkage.

With respect to the locating coupling, robot arm 604 may comprisemounting plate 1216, which may be non-magnetic material, one or moredepressions 1214, lip 1218, and magnets 1220. Magnet 1220 is mountedbelow each of depressions 1214. Portions of clamp 1204 may comprisemagnetic material and be attracted by one or more magnets 1220. Throughthe magnetic attraction of clamp 1204 and robot arm 604, balls 1208become seated into respective depressions 1214. For example, balls 1208as shown in FIG. 9B would be seated in depressions 1214 as shown in FIG.9A. This seating may be considered a magnetically-assisted kinematiccoupling. Magnets 1220 may be configured to be strong enough to supportthe entire weight of end effector 602 regardless of the orientation ofend effector 602. The locating coupling may be any style of kinematicmount that uniquely restrains six degrees of freedom.

With respect to the reinforcing coupling, portions of clamp 1204 may beconfigured to be a fixed ground link and as such clamp 1204 may serve asa five bar linkage. Closing clamp handle 1206 may fasten end effector602 to robot arm 604 as lip 1212 and lip 1218 engage clamp 1204 in amanner to secure end effector 602 and robot arm 604. When clamp handle1206 is closed, spring 1210 may be stretched or stressed while clamp1204 is in a locked position. The locked position may be a position thatprovides for linkage past center. Because of a closed position that ispast center, the linkage will not open absent a force applied to clamphandle 1206 to release clamp 1204. Thus, in a locked position endeffector 602 may be robustly secured to robot arm 604.

Spring 1210 may be a curved beam in tension. Spring 1210 may becomprised of a material that exhibits high stiffness and high yieldstrain such as virgin PEEK (poly-ether-ether-ketone). The linkagebetween end effector 602 and robot arm 604 may provide for a sterilebarrier between end effector 602 and robot arm 604 without impedingfastening of the two couplings.

The reinforcing coupling may be a linkage with multiple spring members.The reinforcing coupling may latch with a cam or friction basedmechanism. The reinforcing coupling may also be a sufficiently powerfulelectromagnet that will support fastening end-effector 102 to robot arm604. The reinforcing coupling may be a multi-piece collar completelyseparate from either end effector 602 and/or robot arm 604 that slipsover an interface between end effector 602 and robot arm 604 andtightens with a screw mechanism, an over center linkage, or a cammechanism.

Referring to FIGS. 10 and 11, prior to or during a surgical procedure,certain registration procedures may be conducted in order to trackobjects and a target anatomical structure of the patient 210 both in anavigation space and an image space. In order to conduct suchregistration, a registration system 1400 may be used as illustrated inFIG. 10.

In order to track the position of the patient 210, a patient trackingdevice 116 may include a patient fixation instrument 1402 to be securedto a rigid anatomical structure of the patient 210 and a dynamicreference base (DRB) 1404 may be securely attached to the patientfixation instrument 1402. For example, patient fixation instrument 1402may be inserted into opening 1406 of dynamic reference base 1404.Dynamic reference base 1404 may contain markers 1408 that are visible totracking devices, such as tracking subsystem 532. These markers 1408 maybe optical markers or reflective spheres, such as tracking markers 118,as previously discussed herein.

Patient fixation instrument 1402 is attached to a rigid anatomy of thepatient 210 and may remain attached throughout the surgical procedure.In an exemplary embodiment, patient fixation instrument 1402 is attachedto a rigid area of the patient 210, for example, a bone that is locatedaway from the targeted anatomical structure subject to the surgicalprocedure. In order to track the targeted anatomical structure, dynamicreference base 1404 is associated with the targeted anatomical structurethrough the use of a registration fixture that is temporarily placed onor near the targeted anatomical structure in order to register thedynamic reference base 1404 with the location of the targeted anatomicalstructure.

A registration fixture 1410 is attached to patient fixation instrument1402 through the use of a pivot arm 1412. Pivot arm 1412 is attached topatient fixation instrument 1402 by inserting patient fixationinstrument 1402 through an opening 1414 of registration fixture 1410.Pivot arm 1412 is attached to registration fixture 1410 by, for example,inserting a knob 1416 through an opening 1418 of pivot arm 1412.

Using pivot arm 1412, registration fixture 1410 may be placed over thetargeted anatomical structure and its location may be determined in animage space and navigation space using tracking markers 1420 and/orfiducials 1422 on registration fixture 1410. Registration fixture 1410may contain a collection of markers 1420 that are visible in anavigational space (for example, markers 1420 may be detectable bytracking subsystem 532). Tracking markers 1420 may be optical markersvisible in infrared light as previously described herein. Registrationfixture 1410 may also contain a collection of fiducials 1422, forexample, such as bearing balls, that are visible in an imaging space(for example, a three dimension CT image). As described in greaterdetail with respect to FIG. 11, using registration fixture 1410, thetargeted anatomical structure may be associated with dynamic referencebase 1404 thereby allowing depictions of objects in the navigationalspace to be overlaid on images of the anatomical structure. Dynamicreference base 1404, located at a position away from the targetedanatomical structure, may become a reference point thereby allowingremoval of registration fixture 1410 and/or pivot arm 1412 from thesurgical area.

FIG. 11 provides an exemplary method 1500 for registration consistentwith the present disclosure. Method 1500 begins at step 1502 wherein agraphical representation (or image(s)) of the targeted anatomicalstructure may be imported into system 100, 300 600, for example computer408. The graphical representation may be three dimensional CT or afluoroscope scan of the targeted anatomical structure of the patient 210which includes registration fixture 1410 and a detectable imagingpattern of fiducials 1420.

At step 1504, an imaging pattern of fiducials 1420 is detected andregistered in the imaging space and stored in computer 408. Optionally,at this time at step 1506, a graphical representation of theregistration fixture 1410 may be overlaid on the images of the targetedanatomical structure.

At step 1508, a navigational pattern of registration fixture 1410 isdetected and registered by recognizing markers 1420. Markers 1420 may beoptical markers that are recognized in the navigation space throughinfrared light by tracking subsystem 532 via position sensor 540. Thus,the location, orientation, and other information of the targetedanatomical structure is registered in the navigation space. Therefore,registration fixture 1410 may be recognized in both the image spacethrough the use of fiducials 1422 and the navigation space through theuse of markers 1420. At step 1510, the registration of registrationfixture 1410 in the image space is transferred to the navigation space.This transferal is done, for example, by using the relative position ofthe imaging pattern of fiducials 1422 compared to the position of thenavigation pattern of markers 1420.

At step 1512, registration of the navigation space of registrationfixture 1410 (having been registered with the image space) is furthertransferred to the navigation space of dynamic registration array 1404attached to patient fixture instrument 1402. Thus, registration fixture1410 may be removed and dynamic reference base 1404 may be used to trackthe targeted anatomical structure in both the navigation and image spacebecause the navigation space is associated with the image space.

At steps 1514 and 1516, the navigation space may be overlaid on theimage space and objects with markers visible in the navigation space(for example, surgical instruments 608 with optical markers 804). Theobjects may be tracked through graphical representations of the surgicalinstrument 608 on the images of the targeted anatomical structure.

FIGS. 12A-12B illustrate imaging devices 1304 that may be used inconjunction with robot systems 100, 300, 600 to acquire pre-operative,intra-operative, post-operative, and/or real-time image data of patient210. Any appropriate subject matter may be imaged for any appropriateprocedure using the imaging system 1304. The imaging system 1304 may beany imaging device such as imaging device 1306 and/or a C-arm 1308device. It may be desirable to take x-rays of patient 210 from a numberof different positions, without the need for frequent manualrepositioning of patient 210 which may be required in an x-ray system.As illustrated in FIG. 12A, the imaging system 1304 may be in the formof a C-arm 1308 that includes an elongated C-shaped member terminatingin opposing distal ends 1312 of the “C” shape. C-shaped member 1130 mayfurther comprise an x-ray source 1314 and an image receptor 1316. Thespace within C-arm 1308 of the arm may provide room for the physician toattend to the patient substantially free of interference from x-raysupport structure 1318. As illustrated in FIG. 12B, the imaging systemmay include imaging device 1306 having a gantry housing 1324 attached toa support structure imaging device support structure 1328, such as awheeled mobile cart 1330 with wheels 1332, which may enclose an imagecapturing portion, not illustrated. The image capturing portion mayinclude an x-ray source and/or emission portion and an x-ray receivingand/or image receiving portion, which may be disposed about one hundredand eighty degrees from each other and mounted on a rotor (notillustrated) relative to a track of the image capturing portion. Theimage capturing portion may be operable to rotate three hundred andsixty degrees during image acquisition. The image capturing portion mayrotate around a central point and/or axis, allowing image data ofpatient 210 to be acquired from multiple directions or in multipleplanes. Although certain imaging systems 1304 are exemplified herein, itwill be appreciated that any suitable imaging system may be selected byone of ordinary skill in the art.

Robot system 300 may include an image registration facility, for exampleas part of computer subsystem 504 and further, for example, as part ofcomputer 406. Registration facility may be specifically configured toperform registration by acquiring and processing patient medical imagesin preparation for a medical procedure (e.g., surgery). The registrationmay be conducted in order to position a medical object in one coordinatesystem relative to another coordinate system, such as betweenpre-operative, intra-operative, and real-time image data of a patient210. A medical object may be a passive implant (e.g., screw, pin),electronics-based implant (e.g., artificial pacemaker, cochlearimplant), bioactive implant (e.g., pharmaceutical implant), biologicaltransplant tissue, artificial transplant material, and the like. Forinstance, image guidance or robot-assisted image guidance may beperformed using a preoperative 3D image dataset such as a computedtomography (CT) scan or magnetic resonance imaging (MRI) scan.Co-registration of multiple coordinate systems may then be needed, suchas between the preoperatively obtained anatomical CT or MM coordinatesystem, an intraoperatively obtained anatomical coordinate system, acoordinate system of the tracking cameras, and the like. Co-registrationof multiple coordinate systems may utilize 2D-3D registration, such aswhere multiple 2D x-ray radiographs of the patient are taken at the timeof surgery, where the position of the x-ray machine and the patient aretracked (e.g., using tracking cameras). The coordinate system in whichthe x-rays were taken may then be registered to the preoperativelyobtained 3D medical image coordinate system through 2D-3D registration.

Co-registration of multiple coordinate systems may involve an iterativeprocess. For example, a 3D CT or MM dataset may be used to generate 2Dreconstructed planar images simulating x-ray radiographs. The generationof 2D reconstructed simulated x-ray images from a 3D dataset maycomprise tracing and integrating the intensities along rays from a pointsource projected through the volumetric medical image on a 2D plane,such as in preparation for generating a digitally reconstructedradiograph (DRR). DRRs may then be generated iteratively until theymatch the actual 2D x-ray images; that is, until the features orintensity characteristics of the bone structures on the DRRs and actualradiographs overlap within some tolerance. In embodiments, initialconditions for the computational configuration may be established inorder to reduce the number of iterations required in this process. Forinstance, a computational configuration may be initialized such thatsimulated and actual x-rays agree to a predetermined level beforestarting iterations, thus potentially reducing the number of iterationsand likelihood for reaching co-registration convergence associated withregistration of the multiple coordinate systems.

Initial conditions for the computational configuration for registrationof multiple coordinate systems may include a step where a user (e.g.,surgeon, doctor, medical technician, medical assistant, and the like),manipulates software to enable graphic objects representing surgicalobjects that he/she intends to implant during a surgical procedure to besuperimposed over the anatomy that appears on multiple intraoperativelyobtained images (e.g., x-ray images). In embodiments, when the graphicobjects are applied, their appearance may be depicted as similar to theappearance of shadows that would appear on x-ray if the surgicalhardware had been implanted and an x-ray then taken. Before or afterplacing the graphic objects on 2D images, the user may also manipulatesoftware such that graphic objects are superimposed on 3D medical imagesin the same anatomical location as the graphic objects applied to the 2Dimages. In an example, preoperative 3D medical images may be usedtogether with intraoperative 2D x-ray radiographs, where the user wouldlikely first (preoperatively) plan screw placement on the 3D medicalimages, then intraoperatively plan the same screws on two or more 2Dx-ray radiographs. For instance, before surgery, the user could use thesystem to plan a pedicle screw at a particular vertebra on 3Dpreoperative CT or MRI images and then plan the same pedicle screw onx-rays by interacting with the system in the operating room beforebeginning surgery. FIG. 13 shows part of the process of planning thetrajectory of a pedicle screw 1602 on a single lumbar vertebra (L3) frommutually orthogonal slices through a computed tomography volume 1600(top left quadrant 1600A, top right quadrant 1600B, and bottom rightquadrant 1600D) and, optionally, a 3D posterior perspective view (bottomleft quadrant 1600C). To plan the screw trajectory, the user manipulatesthe positions of graphic object representing the screw 1602 that isoverlaid on the computed tomography volume image volume 1600 throughinteraction with the system (e.g., through mouse, touchscreen, voicecommand, or other interactive methods). FIG. 14 shows part of theprocess of planning the trajectory of the pedicle screw 1602 on a singlelumbar vertebra (L3) on x-ray views 1700 from an anteroposterior x-rayview 1700A and a lateral x-ray view 1700B. To plan the screw trajectory,the user manipulates the positions of graphic object representing screw1602 that is overlaid on the x-ray images through interaction with thesystem.

As the user may not be able to exactly match the locations of thehardware in the 2D and 3D image sets the user may be enabled to makeplacement within a tolerance range (e.g., within set linear or rationaldimensional limits), within predetermined placement criteria (e.g.,placement constraints stored in a profile for different surgicalobjects, anatomical features, and the like). The system may then use theobject locations in different coordinate systems (e.g., both the CT (orMM) and x-ray coordinate systems) to determine areas of interest on theimages so that the iterative process starts on and focuses within thisregion in attempting to match simulated and actual images.

In embodiments, the system could use as little as one object planned in2D and 3D. However, using two or more planned objects may result in animproved performance since one planned object provides limited degreesof freedom (e.g., 5 of 6 degrees of freedom). That is, with one plannedobject, it may not be clear what the orientation of the anatomy is inits rotational alignment of the object, such as around the shaft of asurgical screw. Using the entirety of the intended construct (e.g.,multiple object placements) has the additional benefit of demarking theentire region of interest, whereas if only a portion of the surgicalhardware construct is used, the algorithm may need to extrapolateoutside of the indicated region to match anatomical features (e.g.,bones) that may or may not need to be accurately targeted.

It may not be immediately clear to the user who has already planned theplacement of objects using 3D medical images on CT or MM where thecorresponding objects should go when planning on 2D planar x-rayradiographs. The reason for the difficulty in correlating the images isthat the 3D medical image planning occurs while the user is looking attwo or three mutually orthogonal slices through the image volume; thatis, only the anatomy on the slice itself may be shown, not anatomy infront of or behind that slice. However, when planning in 2D, the usermay be looking at two or more projections through the same anatomicalfeatures. Additionally, by the nature of x-rays, the projected imagesoriginate from a point source and travel through the patient to an imageintensifier or collector plate through a conical beam, introducingparallax that may be difficult to reconcile mentally by the user whencomparing 2D and 3D images. If the 3D medical image is already presentat the time the 2D images are shot, as it would be if a preoperative CTor Mill is used, DRRs such as those that are generated in a matchingalgorithm may be used as a tool for the user when planning objectplacement on 2D images. That is, the software may be able to generateand display DRRs that the user can quickly adjust to be roughly similarin appearance to the actual x-ray radiographs (e.g., a default may beanteroposterior and lateral x-rays) where software can automaticallysuperimpose the objects that were planned on 3D images onto these DRRs.Given this procedure it may be relatively easy for the user to planobject placement that is similarly placed on the actual 2D x-rayradiographs when side by side with the objects visible on DRRs.

Providing for initial conditions as part of registration may help aniterative matching algorithm to converge more quickly because theapproximate location of object placement is known to the system, such asin both the 2D and 3D images. Although the object positions may notagree exactly because of user placement error, if placed within theconstraints set by the system (or by the user through placementconstraints stored in a profile) may be close enough that the algorithmcan start making small instead of coarse adjustments to converge to asolution. In the alternative circumstance, where no starting position isprovided, the algorithm may start varying the orientation of thesimulated x-rays in the wrong direction, potentially converging on aminimum error value that exceeds tolerance (e.g., where convergence isnot reached).

Another advantage of providing an initial placement on the images isthat the user is able to place objects on actual imaged anatomy ratherthan with reference to identification of landmarks on the anatomy.Identification of landmarks on the anatomy can be challenging formedical personnel because it may be difficult to visualize, such as towhich way bony curvatures travel, especially in 2D views. For example,if the user is asked to mark the outermost extension of a bony processbut the bony process is oriented toward the direction of the x-ray path,it may be unclear what portion of the resulting shadow corresponds tothe outermost extension of the process. However, most users of theprocess should be familiar with the appearance of surgical objects whenoptimally placed on both 2D and 3D views, and should therefore alreadybe familiar with how the objects should appear on medical images.

Initial placement may also automatically fulfill the purpose ofsegmentation sometimes required by algorithms for matching 2D to 3Dmedical images, such as where the user must superimpose ranges (e.g.,boxes) around each anatomical feature (e.g., vertebra) on 2D or 3Dimages, thereby indicating to the software the anatomical level of eachvertebra so that the matching algorithm can be performed independentlyfor each spinal level. The reason for independently registering eachanatomical feature is that the 3D image is generally taken while thepatient is lying supine and the 2D images are generally taken while thepatient is lying prone, and even if the images were taken in roughly thesame orientation, there could be some movement between the features,which would mean that 2D-3D registration at one level would notnecessarily be valid at another level. By setting initial conditionsthrough placement of objects prior to the registration process (e.g.,iterative convergence of multiple coordinate systems), the user knowsthe level where an object is being planned. For example, if a surgeonintends to perform a long construct from LI to LS, he or she may planscrews in all of these levels preoperatively on a 3D image, then againintraoperatively on a 2D image. By performing these planning steps, thesoftware would be provided similar information about the anatomicallevel of each part of the image. For instance, an image analysisalgorithm may then check for a shift in the pixel intensity of the imagewhile moving outward away from the objects that could be used toautomatically segment the edges of each level on 3D and 2D images.Additionally, the known general shape of anatomical features (e.g.,vertebra) and the known general spacing between adjacent objects (e.g.,surgical screws) can be used to improve an automatic algorithm fordetecting edges of the anatomical feature and the object. With referenceto the preceding example, the software may then independently run thematching algorithm on each vertebra from LI to LS to give uniqueregistration for each level.

Wherein the present disclosure utilizes surgical examples such asinsertion of surgical screws into a spinal column (e.g., as depicted inFIGS. 13 and 14), the present disclosure presents methods and systemsfor improving the registration convergence of multiple anatomic imagesutilized in any medical procedure for which medical objects areimplanted into a body. One skilled in the art will appreciate that themethods and system disclosed herein may be applied to any medicalimplant or transplant known in the art.

FIG. 15 illustrates an exemplary method 2500 consistent with theprinciples of the present disclosure. Method 2500 may be performed andused by the robot system as disclosed above. Method 2500 may begin atstep 2502 where the robot may import and store image data from a 3Dimaging system and a 2D imaging system. The system may also store a 3Danatomical feature for a first coordinate system and a 2D anatomicalfeature for a second coordinate system. These imaging systems may be thesame as described above and include a 3D CT scan and a 2D x-ray. Inaddition, the system may store placement information of a digitalmedical object on both the 3D and 2D images. At step 2504, certainfeatures of the intra-op 2D images may be enhanced using imageprocessing. At step 2506, the stored 3D image may undergo a rigidtransformation to generate 2D DRR images (simulated 2D images) withvolume orientation and location information in order to ultimatelyregister the 3D image data to the intra-operative 2D image data. Thevolume and orientation information may be based upon the initial orsubsequent iterations of the simulated 2D images depending on theresults of a comparison that is described below.

At step 2508, features of the simulated 2D image data are enhanced usingimage processing and at step 2510 an initial feature of the simulated 2Dimage data may be compared to an image feature of the intra-operative 2Dimage feature. As a starting point for the comparison, the storeddigital medical image from step 2502 may be used.

At step 2512, the system may determine if a match has occurred of thecurrent feature of the simulated 2D image data and the image feature ofthe intra-operative 2D image data. If a match has occurred, method 2500goes to step 2514 which registers the 3D coordinate system (firstcoordinate system) with the 2D coordinate system (second coordinatesystem). If a match does not occur at step 2512, method 2500 goes tostep 2506 to compare a next iteration or another image feature of thesimulated 2D image data to the image feature of the 2D image data. Thisrepeats until a match has occurred, sending method 2500 to step 1514 toregister the first and second coordinate systems.

While the invention has been disclosed in connection with the preferredembodiments shown and described in detail, various modifications andimprovements thereon will become readily apparent to those skilled inthe art. Accordingly, the spirit and scope of the present invention isnot to be limited by the foregoing examples, but is to be understood inthe broadest sense allowable by law.

All documents referenced herein are hereby incorporated by reference.

What is claimed is:
 1. A surgical robot system comprising: a robothaving a robot arm and a robot base, wherein the robot base comprises animage registration facility configured to receive a three-dimensional(3D) medical image and two-dimensional (2D) medical image; and at leastone camera configured to detect a one or more tracking markers in ananatomical coordinate system; wherein the image registration facility isconfigured to generate a simulated 2D digital medical image from the 3Dmedical image and match the simulated 2D digital medical image to the 2Dmedical image, wherein the image registration facility initiallyassociates the simulated 2D digital medical image to the 2D digitalmedical image by corresponding a first digital medical object placed onthe 3D digital image with a second digital medical object placed on the2D digital medical image, wherein the position of the robot arm and aposition of the patient, using the one or more tracking markers, in theanatomical coordinate system may be represented on a display depictingthe first coordinate system wherein the robot arm is coupled to an endeffector, the end effector being configured with one or more trackingmarkers on the surface of the end effector to be monitored when the endeffector is translated and rotated in the surgical field, wherein theend effector includes a clamp configured to be a ground link and fastenthe end effector to the robot arm, wherein the robot arm includes a ringthat is configured as a visual indicator to notify a user of differentmodes the system is operating under and configured to provide warningsto a user.
 2. The system of claim 1, wherein the 3D digital image isgenerated by at least one of a computed tomography (CT) system andmagnetic resonance imaging (MRI) system.
 3. The system of claim 1,wherein the 2D digital image is generated by an x-ray system.
 4. Thesystem of claim 1, wherein the anatomical coordinate system is at leastin part determined by an image tracking system that tracks the locationof the patient.
 5. The system of claim 1, wherein the 3D digital medicalimage is pre-operatively obtained from one of a tomography (CT) systemand magnetic resonance imaging (MRI) system, and the 2D digital medicalimage is inter-operatively obtained from an x-ray system.
 6. The systemof claim 1, wherein the digital medical object is a surgical medicalimplant device.
 7. The system of claim 1, wherein the digital medicalobject is at least one of a digitally-simulated bio-active implantdevice, electronics-based medical implant device, and artificial medicaltransplant material.
 8. The system of claim 1, wherein the placement ofthe digital medical object appears as a shadow.
 9. The system of claim1, wherein the placement of the digital medical object provides a regionof interest upon which the step of comparing begins.
 10. The system ofclaim 1, wherein the placement of the corresponding digital medicalobject placed the 3D digital image and the acquired 2D digital medicalimage is provided with a placement tolerance.
 11. The system of claim 1,wherein the match is determined within a matching tolerance.
 12. Amethod for registration of digital medical image coordinate systems,comprising the steps of: storing a 3D digital medical image comprising a3D anatomical feature and a first coordinate system; storing a 2Ddigital medical image comprising a 2D anatomical feature and a secondcoordinate system; storing a placement of a digital medical object onthe 3D digital medical image and the 2D digital medical image;determining a registration of the first coordinate system with thesecond coordinate system based on the match, wherein the robot arm iscoupled o an end effector, the end effector being configured with one ormore tracking markers on the surface of the end effector to be monitoredwhen the end effector is translated and rotated in a surgical field,wherein the end effector include a clamp configure to be a ground linkand fasten the end effector to the robot arm, wherein the robot armincludes a ring that is configured as a visual indicator to notify auser of different modes the system is operating under and configured toprovide warnings to a user.
 13. The system of claim 12, wherein the 3Ddigital image is generated by at least one of a computed tomography (CT)system and magnetic resonance imaging (MRI) system.
 14. The system ofclaim 12, wherein the 2D digital image is generated by an x-ray systemand the second coordinate system is in reference to an anatomicalcoordinate system of a patient during a medical procedure.
 15. Thesystem of claim 14, wherein the anatomical coordinate system is at leastin part determined by an image tracking system that tracks the locationof the patient and the x-ray system.
 16. The system of claim 12, whereinthe 3D digital medical image is pre-operatively obtained from one of atomography (CT) system and magnetic resonance imaging (MRI) system, andthe 2D digital medical image is inter-operatively obtained from an x-raysystem.
 17. The system of claim 12, wherein the digital medical objectis a digitally-simulated surgical medical implant device.
 18. The systemof claim 12, wherein the digital medical object is at least one of adigitally-simulated bio-active implant device, electronics-based medicalimplant device, and artificial medical transplant material.
 19. Thesystem of claim 12, wherein the placement of the digital medical objectappears as a replica of a real object on a medical image.
 20. The systemof claim 12, wherein the placement of the corresponding digital medicalobject placed the 3D digital image and the acquired 2D digital medicalimage is provided with a placement tolerance.